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(54) Semi-automated segmentation method for 3-dimensional ultrasound 

(57) The present invention provides a semi -auto- 
mated method for three-dimensional ultrasound for con- 
structing and displaying 3D ultrasound images of 
luminal surfaces of blood vessels. The method com- 
prises acquiring a 3D ultrasound image of a target ves- 
sel and segmenting the luminal surfaces acquired from 
the 3D ultrasound image of the target vessel to gener- 
ate a 3D ultrasound image of the lumen of the target 
vessel, wherein an inflating balloon model is used for 
segmenting the luminal surfaces of the target vessel. 
The method is useful for diagnostic assessment of bod- 
ily vessels as well as provides for therapy planning and 
as a prognostic indicator. 
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Description 

[0001] The present invention relates to a semi-automated method for constructing and displaying 3D ultrasound 
images. In particular, the semi-automated method provides 3D ultrasound images of luminal surfaces of blood vessels. 

5 [0002] The severity of atherosclerosis at the carotid artery bifurcation is correlated with the occurrence of stroke. 1 > 2 
Since most strokes associated with carotid atherosclerosis can be prevented by surgical or non-surgical treatment 3 , the 
identification and monitoring of carotid disease is important for the management of patients at risk of stroke. Currently, 
conventional means of diagnosing and assessing the progression of atherosclerosis involve either determining the 
degree of stenosis with x-ray angiography or MR A, or with techniques which are sensitive to abnormalities in blood flow 

10 rate, such as Doppler ultrasonography. Unfortunately, none of these known techniques provide a clear three dimen- 
sional image of the target tissue in a manner which allows for the non-invasive and detailed view of the blood vessels. 
Moreover, none of the known techniques provide detailed visualization of the lumen surfaces of blood vessels within the 
body. 

[0003] It is therefore an object of the present invention to provide a novel method, based on 3D ultrasound, which 
15 is non-invasive and provides detailed three dimensional views of internal luminal surfaces of blood vessels. An impor- 
tant task in the development of such a technique is the segmentation (i.e., extraction) of vessel surfaces from ultrasound 
images for the purposes of visualization, therapy planning, and volumetric measurements. Deformable surface models 
have become particularly useful tools for this method. 4 

[0004] According to one aspect of the present invention there is provided a semi-automatic 3-D ultrasound segmen- 
20 tation method for displaying luminal surfaces of vessels, the method comprising the steps of: 

acquiring a 3D ultrasound image of a target vessel; and 

segmenting the luminal surfaces from said 3D ultrasound image of said target vessel to generate a 3D ultrasound 
image of the lumen of said target vessel; wherein an inflating balloon model is used for segmenting the luminal sur- 
25 faces of said target vessel. 

[0005] In another aspect of the present invention, there is provided a method for the diagnosis and prognosis of 
vessel disease, said method comprising the steps of: 

30 - acquiring a 3D ultrasound image of a target vessel; 

segmenting the luminal surfaces from said 3D ultrasound image of said target vessel to generate a 3D ultrasound 
image of the lumen of said target vessel using an inflating balloon model; 

inspecting the generated 3D ultrasound image of the lumen of said target vessel to assess the presence of a 
shadow representing a disease lesion within said lumen. 

35 

[0006] Embodiments of the present invention will now be described, by way of example, with reference to the 
accompanying drawings in which 

Figure 1 illustrates the subdivision of a mesh used in the inflating balloon model, Figure 1(a) shows the edge 1-3 
40 exceeds the user-specified threshold length while Figure 1 (b) shows the edge 1-3 is divided into two with new tri- 
angles being formed; 

Figure 2 illustrates two views of a primitive shape consisting of a cylinder and two hemispherical end-caps used to 
generate a 3D volume of interest (VOI); 

45 

Figure 3 illustrates a 3D volume of interest (VOI) constructed from five primitive shapes, each basic shape is scaled 
and stretched to fit approximately into the carotid vessel; 

Figure 4 illustrates a section through a 3D ultrasound image of diseased carotid arteries showing a plaque (P) on 
so the right-hand side of the image, the internal carotid (ic), external carotid (ec) and common carotid (cc) arteries are 
labeled; 

Figure 5 illustrates a segmented boundary for the free-hand 3D ultrasound image as made by the method disclosed 
herein and as shown in Figure 4. the mesh is shown superimposed on three orthogonal slices through the ultra- 
. . 55 sound image; 

Figure 6 illustrates the segmented boundary of the image of Figure 4 shown as a rendered surface, the internal 
carotid (ic), external carotid (ec), common carotid (cc) and carotid bulb are indicated; and 



BNSDOCIO. <EP 1 03019 1A2 I > 



2 



EP 1030 191 A2 



Figure 7 illustrates a variance map for segmentation of a carotid vessel from a 3D ultrasound image, showing high 
variability in mesh portion at the .apex of the bifurcation, Figure 7(a) and Figure 7(b) show two orientations of the 
same vessel. 

5 [0007] According to an embodiment of the present invention, there is provided a 3D semi-automatic segmentation 
method, based on a deformable model, for extracting and displaying the lumen surfaces of vessels from 3D ultrasound 
images. The method uses a deformable model which first is rapidly inflated to approximate the boundary of the artery, 
the model is then further deformed using image-based forces to better localize the boundary. The method can be used 
in the diagnosis and prognosis of various diseases associated with blood vessels such as atherosclerosis. 

io [0008] The method requires that an operator selects an arbitrary position within a target vessel, such as for exam- 
ple a carotid vessel, as a starting point for the development of the model. Since the choice of initialization position 
affects the subsequent development of the deformable model, there is variability in the final segmented boundary. The 
performance of the segmentation method has been tested by examining the local variability in boundary shape as the 
initial position is varied throughout the lumen of a 3D ultrasound image of a carotid bifurcation. The test results indicate 

is that high variability in boundary position occurs in regions where either the segmented boundary is highly curved or the 
3D ultrasound image has no well defined vessel edges. 

Acquisition of 3DUItrasound Images 

20 [0009] A freehand imaging system 5 - 6 was used to acquire 3D ultrasound images of target vessels for use in the 
present method. The system makes use of a six degree of freedom DC magnetic field-based POM device (Flock of 
Birds, Ascension Technologies) affixed to an ultrasound probe to rack the position and orientation of the probe during 
the scan. Video frames from an ultrasound machine (Ultramark-9, Advanced Technology Laboratories) were digitized 
with a video frame grabber (RasterOps 24XLTV, Truevision) and saved to a Macintosh computer along with simultane- 

25 ous recordings of the transducer orientation and position. Image reconstruction involves transforming each input image 
by a matrix which is determined from the recorded probe orientation during image acquisition. Applying the transforma- 
tion matrix to each image converts the image to a global coordinate space. Once the images have been transformed, 
the spatial extent and resolution of the output 3D image is determined. The final 3D ultrasound image is reconstructed 
one slice at a time by determining the nearest input image pixel for each voxel. Gaps between image planes are filled 

30 in using nearest neighbor interpolation. 6 The final 3D ultrasound image was then used as the basis for the segmenta- 
tion of the target vessel surface. 

Segmentation 

35 [0010] An inflating balloon mode! forms the basis of the present algorithm for segmenting the target vessels. 
Although the present method can be used to examine luminal surfaces of any fluid filled tissue, for the purposes of illus- 
tration, carotid arteries were used as the choice of vessel in the present method. The algorithm consists of three major 
steps: (1) interactive placement of the initial balloon model inside the lumen of the artery; (2) automatic inflation of the 
model towards the arterial wall; and, (3) automatic localization of the arterial wall. The balloon model is represented by 

ao a closed mesh of triangles, with the initial mesh being an icosahedron. After the initial model is placed inside the artery, 
it is rapidly inflated towards the arterial wail. When equilibrium under the influence of inflation forces is reached, the 
model approximately represents the shape of the artery. This approximate mesh is then further deformed by means of 
image-based forces to localize the wall of the artery. Equations describing the dynamics of the model and the forces 
acting on it are provided below. 

45 

(a) Dynamics 

[001 1 ] The equation of motion for vertex i of the mesh is given by 8 

so m t x,(t) + v/xfit) + g(x / (f)) -f(x,(f)) (1) 

where x,(f) is the position of the vertex, x ,(f) and x ,(f) are its velocity and acceleration, respectively, m- t is its mass, v t 
is the damping coefficient, g,{*/(0) is the resultant surface tension at the vertex and f,(x,(f)) is a "driving" force. Equilib- 
rium is reached when both x >(r) and x ,(f) become zero, which can take a very long time. 7 By setting the mass of each 
55 vertex to zero and the damping coefficient to unity. Equation (1) is reduced to: 

x / (/)=f(x / (0)-g(x ,-(/)) (2) 
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The system represented by Equation (2) reaches equilibrium when x,(f) becomes zero, which can occur quickly since 
the system does not possess inertia. To compute the equilibrium position of each vertex, and hence the deformed 
shape of the model, Equation (2) is iteratively updated from time f to time t + Af using the formula: 

5 x,(f + Ay * xfi) + fffx/y; - gfx/^At (3) 

Iterations continue until x /(f) becomes approximately zero for ail vertices. After each iteration, the triangles of the mesh 
grow m size. In order to model accurately the shape of the artery, the triangles of the mesh are subdivided to form 
smaller triangles. Essentially, after each iteration, the length of each edge in the mesh is compared to a user-specified 
10 threshold length. If the length exceeds the threshold, the edge is divided into two edges of equal length and the two tri- 
angles on either side of the original edge are replaced by four triangles as shown in Figure 1. 

(b) Driving Forces 

15 [001 2] In step (2) of the algorithm, the model is driven towards the arterial wall by means of an inflation force acting 
on each vertex /: 



20 



SO 



( klnM*)> if /(*#)) * Tmd x,<0 e VOI (4) 
flnfaiif)) = | q otherwise 



25 where /c /n/ is the amplitude of the inflation force and n,(f) is the normal at the vertex, which is computed as the average 
of the normals of each triangle attached to the vertex. The inflation force is turned on at vertices where the correspond- 
ing image intensity, /(x,(f)), is below a user-selected gray-level threshold T. In practice, the balloon model may leak out- 
side the arterial wall since artifacts such as accoustic shadows result in large voids adjacent to the artery, which have 
the same mean gray level as the lumen. The occurrence of a calcified lesion on the lumen of the carotid vessels is one 

30 such cause of shadow. To prevent the model from growing too far beyond the arterial wall, the inflation force is only 
applied for points which are inside a user-defined volume of interest (VOI). The VOI is modelled as the union of simple 
geometric shapes, constructed from cylinders with rounded ends. The complete VOI is formed by joining five such 
shapes, individually sized and oriented to fit the carotid vessel. 

[001 3] A single geometric primitive is shown in Figure 2 and the complete VOI used in this study is shown in Figure 
35 3. The boundary obtained by the application of an inflation force alone is an approximation to the actual wall of the 
artery since it is difficult to identify a threshold, which exactly separates points inside the lumen from those outside and 
because of the presence of artifacts. In step (3) of the algorithm, this approximation is refined. To localize the arterial 
wall better a 3D potential function (P) is constructed from the image data which attracts the model to 3D intensity edges 
(gradients): 

40 

P(X ' (0) = I(vgJ/), + e <S 

45 where is a 3D Gaussian smoothing filter 9 of characteristic width a and e is a small constant to prevent division by 
zero. Minima of the potential function coincide with the arterial wall. The potential function produces a force field which 
is used to deform the model: 



f edge (*/<0) = -A«*e*P(x,(0> (6) 
where controls the strength of the force, 
(c) Surface Tension 

55 [0014] Surface tension keeps the model smooth in the presence of noise. It is simulated by treating each edge of 
the model as a spring. Surface tension at vertex / is computed as the vector sum of each normalized edge vector con- 
nected to the vertex: 
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k M 

n tens 



40 



45 



55 



5 where e,y = x / (f)-x / (f) is a vector representing the edge connecting vertex / to an adjacent vertex j, /c tens controls the 
strength of the resultant tensile force, and M is the number of edges connected to vertex /. 

Assessment of Variability 

10 [001 5] Accuracy is often used as the basis for assessing the performance of a segmentation method. 10 " 12 However, 
in cases where the segmented boundary depends on some form of operator initialization, boundary variability also 
plays an important role in judging performance. The present method requires that an operator selects an initialization 
position within the carotid vessel, which affects the final segmented boundary. Variability in the position and shape of 
the final segmented boundary occurs because the user-defined initialization position is arbitrary. For applications such 

15 as monitoring the progression of atherosclerosis over time or its regression due to non-surgical treatment, low variability 
assessment of the vessel boundary is required. Furthermore, assessment is particularly crucial at local regions with 
lesions. For this reason, a local measure of variability has been selected as a means of assessing the performance of 
our segmentation method. Only the variability associated with the choice of initialization position is considered and all 
other segmentation parameters remain constant throughout. The procedure involves: (1) generating an ensemble of 

20 meshes, which span the space of likely initialization positions; (2) determining an average segmentation boundary from 
the set of meshes; (3) determining a description of the spatial distribution of the meshes; and, (4) computing the vari- 
ance of mesh locations on the surface of the average boundary. The steps involved in determining our measure of local 
variability are described in the following sections. 

25 (i) Generating an Ensemble of Meshes 

[0016] An ensemble of segmentation meshes is generated using different initialization positions for each mesh. The 
positions art spaced evenly throughout the interior of the carotid vessel. A point, x, is considered to lie within the vessel 
if it satisfies both conditions in equation (4), namely, /(x) <, T and x £ VOL For the 3D ultrasound image shown in Figure 
30 4, there are more than half a million voxel positions which satisfy these conditions. In order to reduce the required com- 
puting time for determing the ensemble of meshes, the set of suitable positions are subsampled by a factor of nine in 
each dimension, resulting in about 750 meshes. On a 600 MHz Compaq Alpha workstation, mesh generation for all 750 
positions takes about three hours. 

35 (ii) Generating the Average Mesh 

[0017] The procedure for determing the average of the ensemble of meshes is based on establishing a one-to-one 
correspondence between points in one mesh and points in every other mesh. The set of N meshes, 



each have M N points. A mesh 



was randomly selected, and for each point x# in 
so XL 

the closest points x i2 . x#, • • • , x, w in meshes 

AS>Ai»- * »A3V 

are found. A point, y, on the average mesh Y is found by determining the centroid of ail corresponding points x,y , x i2 , 
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(Hi) Generating the Boundary Density Function 

[0018] In order to compute the statistical parameters associated with the variability of the 3D segmentation, a dis- 
io crete 3D representation of the spatial distribution of the ensemble of meshes is first defined, which is called the bound- 
ary density function (BDF). The BDF has the same dimensions as the original ultrasound image and each voxel in the 
ultrasound image corresponds to a voxel in the BDF. Each voxel in the BDF is assigned a value which is equal to the 
number of segmented boundary meshes which intersect the equivalent voxel. 

15 (iv) Variance Mao as a Local Measure of Variability 

[0019] Having both an average mesh, and a statistical description of the spatial distribution of meshes (i.e., the 
BDF),the variability in the segmentation algorithm can be determined. For each point y, on the average mesh, Y, the 
BDF is sampled at U discrete points along a line segment centered about y„ and oriented in the direction of the surface 
20 normal, n(y,). This sampling procedure reduces the BDF into a set of one dimensional distributions, D,(l/). Given a 
mesh point, y /( and sampled BDF distribution for the point, D,(i/), the variance V f of the sampled distribution at each 
point y, on the average mesh is: 



Acquisition of the 3D Segmentated Ultrasound Image 

[0020] Figure 4 shows a slice through a freehand 3D ultrasound image of a carotid bifurcation oriented approxi- 
mately in the plane of the bifurcation. Ultrasound insonation occurred from the left-hand side of the image. A plaque is 
35 visible on the right-band wall of the common carotid artery, and casts a shadow on the vessel wall, proximal to the bifur- 
cation. Image values in the shadow portion of the ultrasound image are similar to those found within the carotid vessel, 
therefore, image value alone cannot be used to localize the boundary of the carotid lumen. 

[0021] Figure 5 shows a segmented boundary generated from the freehand 3D ultrasound image shown in Figure 
4. Three orthogonal slices through the ultrasound image are also shown. Figure 6 shows the same segmented bound- 
40 ary from a different orientation, illustrating that the plaque has reduced the diameter of vessel lumen near the carotid 
bulb. 

[0022] Figure 7 shows the local variability in the segmented boundary, determined by sampling the corresponding 
BDF about each point of the average boundary. In this figure, the average mesh is shown with shading which represents 
the local variance in mesh position, determined from the BDF. Dark regions correspond to high variability, while lighter 
45 regions have low variability. Figure 7(a) shows that the apex of the bifurcation is a site of high variability, while 7(b) 
shows that there are other local regions of increased variability. The region corresponding to the calcified plaque 
appears to have low variability in mesh localization. 

[0023] The method of the present invention is relatively simple and requires low computational overhead and thus 
segmentation occurs quite rapidly. A typical 256 x 256 x 256 voxel ultrasound image can be segmented in approxi- 
so mately 10 seconds on a modern workstation, once initial loading and filtering of the ultrasound image has been com- 
pleted. However, the speed of segmentation requires significant computer resources in terms of memory storage. In 
order to gain speed, the entire volume is kept in main memory throughout the segmentation process. A Gaussian fil- 
tered copy of the image also is computed, followed by a three-component force vector field. 

[0024] Image noise in the ultrasound image can prevent the deformable model from reaching the boundary of the 
55 target carotid artery during the stage when driving forces are applied to each mesh point. Spuriously high grey-level val- 
ues (i.e.. /(x,{f)) > 7) will cause mesh points to stop moving prematurely. 

[0025] The results show high variability in segmentation position is associated with regions of ultrasound image 
where the vessel boundary is not well defined (i.e.. low contrast) and with regions of high curvature (such as at the apex 



25 
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and represents the measure of local variability in the segmentation procedure. 
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of the bifurcation). The plaque boundary is well defined and thus the segmented boundaries have low variability in this 
region of the ultrasound image. Since user intervention in initializing the segmentation method results in variability in 
the segmented boundary shape, it is important to assess the performance of the segmentation method based on local 
variability. The present segmentation method based on a free-hand 3D ultrasound image of the carotid vessels, shows 

5 a correlation between a mapping of the local variability in segmentation as a function of initial model position. 

[0026] In summary, a semi-automatic method for extracting and displaying the lumen of vessels from 3D ultrasound 
images has been developed. Such method allows for the rapid and simple non-invasive and clear visualization of the 
internal luminal surfaces of vessels. Specifically, the method allows for the diagnosis, prognosis and treatment of differ- 
ent types of disease lesions within body vessels. 

10 [0027] It is understood by one skilled in the art, that the method of the present invention can be used to provide 3D 
images of any luminal surface of any type of body channel that carries a biological fluid. That is, any fluid filled body 
tissue can be diagnostically analyzed using the method of the present invention, including for example tissue abscesses 
such as kidney or follicular. Body channels include for example but are not limited to vessels such as blood, lymph and 
lacteal as found in humans and/or animals. The method is particularly useful for the management and treatment of arte- 

75 rial disease (i.e. blood vessel) such as arterial atherosclerosis. 

[0028] Variations and modif ications of the present invention should be apparent to those of skill in the art without 
departing from the scope of the present invention as defined by the appended claims. 
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Claims 

1 . A semi-automatic 3D ultrasound segmentation method for displaying luminal surfaces of vessels, the method com- 
prising the steps of: 

acquiring a 3D ultrasound image of a target vessel; and 

segmenting the luminal surfaces from said 3D ultrasound image of said target vessel to generate a 3D ultra- 
sound image of the lumen of said target vessel; wherein an inflating balloon model is used for segmenting the 
luminal surfaces of said target vessel. 

2. The method of claim 1 , wherein said inflating balloon model is represented by an algorithm comprising the steps of: 

interactive initial placement of a balloon model with the target vessel, 
inflation of the model towards the target vessel wall; and 
localization of the target vessel wall. 

3. The method of claim 2, wherein said model is inflated within a user-defined volume of interest comprising a union 
of simple geometric shapes constructed to fit the target vessel. 

4. The method of claim 3, wherein said balloon model is represented by a mesh of triangles placed within the target 
vessel. 

5. The method of claim 4, wherein said model is inflated until said model represents the shape of the target vessel 
and then further deformed by means of image-based forces to localize the wall of the target vessel. 

6. The method of claim 1 , wherein said method further comprises the step of assessing the variability of the boundary 
of the target vessel. 

7. The method of claim 6, wherein assessing the variability comprises the steps of: 

generating an ensemble of meshes which span the space of selected initial balloon model placement; 
determining an average segmentation boundary from the set of meshes; 
determining a description of the spatial distribution of the meshes; and 
computing the variance of mesh locations on the surface of the average boundary. 

8. The method of claim 1 , wherein said target vessel comprises any internal human or animal body channel that car- 
ries a biological fluid. 

9. The method of claim 8, wherein said vessel is selected from the group consisting of blood, lymph and lacteal. 

1 0. The method of claim 1 , wherein said 3D ultrasound image is acquired using a freehand imaging system comprising 
a band-held device affixed to an ultrasound probe to track the position and orientation of the probe during scanning 
of the target vessel. 

1 1 . The method of claim 1 0, wherein digitized data from the scanning is reconstructed into a 3D image by the steps of: 

acquiring 2D images of the target vessel represented by a array of pixels; 

transforming the 2D image array by a matrix to a global coordinate space into a volumetric image array com- 
prising image slices; and 
reconstructing a 3D image one slice at a time. 

12. The method of claim 3. wherein nearest neighbor interpolation is used to fill gaps between image planes. 

13. A method for the diagnosis and prognosis of vessel disease, said method comprising the steps of: 

acquiring a 3D ultrasound image of a target vessel; 
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segmenting the luminal surfaces from said 3D ultrasound image of said target vessel to generate a 3D ultra- 
sound image of the lumen of said target vessel using an inflating balloon model; 

inspecting the generated 3D ultrasound image of the lumen of said target vessel to assess the presence of a 
shadow representing a disease lesion within said lumen. 

5 

14. The method of claim 13, wherein said target vessel is a blood vessel. 

15. An apparatus for 3D ultrasound segmentation comprising means for acquiring a 3D ultrasound image of a target 
vessel; means for segmenting the luminal surfaces from said 3D ultrasound image of said target vessel to generate 

10 a 3D ultrasound image of the lumen of said target vessel, wherein an inflating balloon model is used for segmenting 
the luminal surfaces of said target vessel, and means for displaying said image. 



15 



20 



25 



30 



35 



40 



45 



50 



55 



9 




BNSOOCID: <EP 1030191 A? I > 



EP1 030191 A2 





11 



EP1 030191 A2 




12 



EP 1 030 191 A2 




13 



EP1 030191 A2 




BNSDOCID: <EP 1030191A2 I > 



14 



EP1 030191 A2 




VD 



Cm 



15 

RM.c-nnr.irv <-pp kwmoiao i -» 




BNSOOCID: <EP 1030191 A2 I > 



16 



(19) 



J 



(12) 



Europaisches Patentamt 
European Patent Office 
Office europeen des brevets 



(11) 



EP 1 030 191 A3 



EUROPEAN PATENT APPLICATION 



(88) Date of publication A3: 

16.04.2003 Bulletin 2003/16 

(43) Date of publication A2: 

23.08.2000 Bulletin 2000/34 



(51) mtci7: G01S 15/89, G01S 7/52, 
A61B8/00, G06T 17/00, 
G06T 5/00 



(21) Application number: 00301267.1 

(22) Date of filing: 17.02.2000 



(84) Designated Contracting States: 

AT BE CH CY DE DK ES Fl FR GB GR IE IT LI LU 
MC NL PT SE 

Designated Extension States: 
AL LT LV MK RO SI 

(30) Priority: 19.02.1999 US 253385 

(71 ) Applicant: The John P. Rob arts Research Institute 
London, Ontario N6A 5K8 (CA) 

(72) Inventors: 

• Ladak, Hanif M. 

Brantford, Ontario N3S 5H1 (CA) 



(54) 

(57) The present invention provides a semi-auto- 
mated method for three-dimensional ultrasound for con- 
structing and displaying 3D ultrasound images of lumi- 
nal surfaces of blood vessels. The method comprises 
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segmenting the luminal surfaces acquired from the 3D 
ultrasound image of the target vessel to generate a 3D 
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ultrasound image of the lumen of the target vessel, 
wherein an inflating balloon model is used for segment- 
ing the luminal surfaces of the target vessel. The method 
is useful for diagnostic assessment of bodily vessels as 
well as provides for therapy planning and as a prognos- 
tic indicator. 
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